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Abstract 

One of the most direct human mechanisms of promoting cooperation is rewarding it. We 
study the effect of sharing a reward among cooperators in the most stringent form of social 
dilemma, namely the Prisoner's Dilemma. Specifically, for a group of players that collect 
payoffs by playing a pairwise Prisoner's Dilemma game with their partners, we consider 
an external entity that distributes a fixed reward equally among all cooperators. Thus, in- 
dividuals confront a new dilemma: on the one hand, they may be inclined to choose the 
shared reward despite the possibility of being exploited by defectors; on the other hand, 
if too many players do that, cooperators will obtain a poor reward and defectors will out- 
perform them. By appropriately tuning the amount to be shared a vast variety of scenarios 
arises, including traditional ones in the study of cooperation as well as more complex sit- 
uations where unexpected behavior can occur. We provide a complete classification of the 
equilibria of the «-player game as well as of its evolutionary dynamics. 
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1 Introduction 



Selfish behavior seems to be one of the consequences of evolutionary dynamics. 
Genes, organisms, generic entities acting in their own benefit do better in a strug- 
gle for reproductive (understood in a wide sense) success and are selected in the 
long term. In spite of this general trend, we find in every evolutionary context (be it 
biological, sociological, economic, etc.) many instances in which cooperative be- 
haviors are evolutionarily successful. The explanation of this puzzle has developed 
into an active line of research, and providing a com plete answer to it is one of the 
big open problems of XXI century (IPennisil |2005|) . Many mechanisms have been 
identifie d as responsible f or these coope rative associations. Arnong th em we find 
kinsh ip (iHamilton , 1964a'.^. reciprocity ( Axelrod and HamiltonUl98lh. reputation 
gain (INowak and Sigm und. 1998), and others (|Axelro d. 1984; Nowak. 2006) . One 
of the most interesting mechanis ms of this kind that h as been identified is altru- 
istic punishnient an d rewarding (|Sigmund et alx 1200 1|) or voluntary participation 



(IHauert et alX. |2007|) . Through this mechanism social groups that are engaged in 



social dilemmas, such as the one represented b y the Public G oods game, can over- 
come the well-known tragedy of the commons (lHardinl . ll968[) . 



The rewarding mechanisms just mentioned are of the bottom-up type, i.e., they 
arise at the individual level and lead to cooperation at the group level. However, 
in ecological and social contexts, there are several levels of organization which 
make possible top-down approaches. For instance, parents, educators, governments 
and other institutions promote prosocial behavior by rewarding individuals in dif- 
ferent manners (prizes, incentives, tax deductions, etc.). In biological or ecologi- 
cal contexts, some species reward symbionts t hat coopera t e at t he required level 
by providing them with more resources (see ' Kiers et al. ] (12003 ) and references 
therein). Companies also use similar mechanisms in their own benefit to induce 
custo mers to supply useful infor mation about consumption habits or social net- 
works (|lribarren and Moroll2007|) . Finally, another in stance of top-down rewardin g 
can be found in team form ation of animal soc i eties (lAnderson and Franksl . 1200 lb . 
e.g. in cooperative hunting (IPacker and Ruttanlll988h . 



Top-down rewarding mechanisms can be generically implemented in two different 
ways. The simplest one is to provide a fixed benefit to every cooperator. In terms of 
game theory, this is tantamount to shifting the payoff matrix by a constant added to 
entries related to cooperation. Thus, for instance, if one starts off with a Prisoner's 
Dilemma (PD) to model the baseline social behavior, in troducing such a reward 
transforms the dilemma into anothe r one, either Sno wdrift (|Maynard-Smith and G. Pried . 
19731 : ISugdenl . Il986h or Stag Hunt (|Skyrmsl |20()3|), or even suppresses completely 
the dilemma, changing it into a Harmony game (lLichti . ll999h . A second, more sub- 
tle mechanism is to distribute a fixed amount between all cooperators in the popula- 
tion. In this case, the original PD becomes a new dilemma, because there is a clear 
incentive to cooperate but if there are too many cooperators the incentive disap- 
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pears and hen ce defecting pays. This is reminiscent of the Minority game paradigm 



(|Morol 120041) and, in fact, it may be seen as an alternative form of describing situ- 
ations in which being in the minority (understood in a lax sense) is the best option. 
We will refer to this situation as the shared reward dilemma. 

In this work we study the shared reward dilemma by considering an interaction 
group of n individuals. In order to understand it in the most stri ngent form of social 
dilemm a, interaction among individuals follows the PD (see l Ooebeli and Hauert 



(|2005|) for a review). Thus, we introduce a game in which payoffs can be obtained 



from two sources: first, all players collect payoffs by playing a n-pl ayer general 



ization of the PD game with their partners ( Hauert and Szabd I2003|) . and second. 



players who have chosen to cooperate share an extra payoff coming from a pool. 
In the next section we analyze in detail the n-player game. Situations in which 
multiple interior equilibria occur are completely determined, as well as the para- 
metric settings in which equilibria increase, decrease or jump discontinuously with 
the reward. In Section |3] we analyze the evolutionary stability of the equilibria dis- 
cussed in Section [2] and provide the different asymptotic scenarios of cooperation 
according to the replicator dynamics. Section |4] summarizes our conclusions and 
presents some future prospects. Appendix lAl contains the main mathematical re- 
sults on which the discussions of previous sections rest: a theorem and a corollary 
that provide closed formulae for the symmetric Nash equilibria in terms of the 
reward for finite and large number of players, respectively. To complete our analy- 
sis, we present in Appendix|B]a theorem which characterizes all asymmetric Nash 
equilibria in pure strategies of the game. 



2 The shared reward dilemma 



Consider an assembly of n players, each of whom can choose one out of two ac- 
tions: cooperate (C) or defect (D) with the rest of the n—l players in an one-shot 
game (i.e., all player's actions are simultaneously performed). Players collect pay- 
offs according to a PD game from every one of the n—\ opponents. In addition, 
players who have chosen to cooperate obtain an extra payoff coming from a fixed 
reward p, provided by an external source, that is evenly distributed among all co- 
operators. 

To provide the strategic form of this game we introduce some notation. Let k be the 
number of cooperators in the group. Payoffs of pairwise interactions are denoted 
by the standard parameters of the PD game: a defector that exploits a cooperator 
obtains the temptation T , but when she faces up another defector she receives the 
punishment P; instead, the payoff for a cooperator meeting another cooperator is 
the reward R (not to be confused with p, the reward to be shared that we propose in 
this work), but obtains the sucker's payoff S when she confronts a defector. For the 
game to be a PD, the payoff must be ordered according toT > R> P > S. Since the 
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game is symmetric, in the sense that the payoff to a particular player is independent 
of her label and only depends on her actions, the total payoff of an arbitrary player 
is given by 

{{k—l)R+{n — k)S + ^, if she cooperates, 
kT + {n-l- k)P, if she defects. 

The remaining of this section is devoted to study the Nash equilibria of this game. 

Let us begin with the symmetric Nash equilibria in pure strategies, which can be 
easily obtained from ([U). Full cooperation is an equilibrium if no player increases 
her payoff by defecting unilaterally, that is, if and only if T(n — I) < (n — l)R + 
p/n. Similarly, full defection is an equilibrium if no player increases her payoff by 
cooperating unilaterally, i.e., if and only if (n — 1)5+ p < (n — l)P. The former 
constraint on p suggests a normalization of the shared reward, namely 

(2) 



n{n-\){T-R)' 

which will henceforth be referred to as scaled reward. With this parameter, the 
condition for full cooperation to be a Nash equilibrium is simply 5 > 1 . As for the 
second constraint, if we introduce a new parameter, the defection ratio 

the condition for full defection to be a Nash equilibrium is 5 < 1/«C- the analy- 
sis of the game can be performed solely in terms of these two parameters instead of 
the five parameters that originally define the game. As we have shown, the scaled 
reward is the ratio between the actual reward and the reward needed for full co- 
operation to be a Nash equilibrium; as for the defection ratio, it compares, in a 
pairwise interaction, the excess of payoff a defector gets over a cooperator when 
both confront a cooperator, with that when both face up a defector. 

Note that both full defection and full cooperation will coexist if and only if 1 < 5 < 
l/nC,. Clearly, no reward meets this condition unless ^ < l/n. Thus we see that, by 
increasing the reward, the symmetric Nash equilibrium in pure strategies changes 
from full defection to full cooperation, and in between these two extremes there 
may be either coexistence or absence of both equilibria, depending on whether C, is 
smaller or larger than l/n, respectively. 

The space of symmetric mixed strategies Nash equilibria consists of all < ^ < 1 
such that a player cooperates with probability q and defects with probability l—q. 
The expected total payoffs of an arbitrary cooperator and of an arbitrary defector 
when the rest of the players play an equilibrium q, are given by 
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/c(^) =IE[i7 1 she cooperates] = {n - l)qR+ {n- - q)S + PfJn-i{q), (4) 
/z)(^)=E[t/|shedefects] = (n-l)^r + (n-l)(l-^)/', (5) 

where fJmil) = ^[{Sm + '^)^^], being a binomial random variable which is the 
sum of m i.i.d. Bernoull i 's rand om variables with mean q. As has been observed by 
Chao and StrawdermanI (|l972() . /Um{q) has the expression 



1, 

l-(l-g) 

{m+l)q 



m+l 



for ^ = 0, 
for < ^ < 1 . 



(6) 



Symmetric Nash equilibria in completely mixed strategies can be computed by 
solving fciq) = foil)- To do that, it is convenient to distinguish when there are 
more than two players and when there are just two players involved. The latter 
case is particularly simple because it repro duces the maj or binary games used in 
the study of cooperation. The payoff matrix dGintisl llOOQ i) of this binary game can 
be easily obtained from ([U) by setting n = 2, and it is shown in Table \T\ Thus, 
depending on p, the game becomes a: 

(i) Prisoner's Dilemma, if T > R + p/2 and P > S + p; 

(ii) Snowdrift, if T > R + p/2 and P < S + p; 

(iii) Stag-hunt, if T < R + p/2 and P > S + p; 

(iv) Harmony, if T < R + p/2 and P < S + p. 





C 


D 


c 


R + p/2 


S + p 


D 


T 


P 



Table 1 

Payoff matrix for the binary case of the shared reward dilemma. 

The Nash equilibria of these games are well known. Thus, the Snowdrift game has 
two asymmetric Nash equilibria in pure strategies, {(C,D), (D,C)}, while the Stag- 
hunt game has two symmetric Nash equilibria, {(C,C), (D, D)}. Both games have 
a unique Nash equilibrium in mixed strategies q E (0, 1). Otherwise, the Prisoner's 
Dilemma and the Harmony game have just one Nash equilibrium (both players 
defecting and both cooperating, respectively). 

Li terms of 6 and ^, the above conditions (i)-(iv) can be rephrased as 

(i') Prisoner's dilemma if 5 < min(l, 1 /2Q; 
(ii') Snowdrift if 1/2^ < 5 < 1; 
(iii') Stag-hunt if 1< 5 < 1/2^; 
(iv') Harmony if 5 > max(l, 1/2Q. 

In general, our results permit to characterize the changes in the structure of equi- 
libria by varying 6 and fixing ^. Therefore, we can study the effect of rising the 
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Fig. 1. Symmetric Nash equilibria of the binary game as a function of the scaled reward 5 
for the two types of possible behavior, ^ > 1/2 (left) and ^ < 1/2 (right). 

reward. In order to illustrate our approach, consider once more the binary game. 
Upon increasing 5 the game changes from Prisoner's dilemma to Harmony. For 
^ = 1/2 this change occurs directly when 5 crosses at 1, but depending on whether 
C,> 1/2 or ^< 1/2, the change occurs via Snowdrift or via Stag-hunt, respectively. 

Taking n = 2 in (HI) and dS]) (hence /Ji (^) = 1 — q/2) and solving /c(<?) = foici) we 
obtain a unique Nash equilibrium in mixed strategies < q < I given by 



1-25C 
1-(1+5)C' 



(7) 



If ^ > 1/2 (respectively ^ < 1/2) ^ is a continuous increasing (respectively decreas- 
ing) function of 5. Figure [Hillustrates these two scenarios as well as the parametric 
conditions for the existence and coexistence of equilibria in pure strategies. When 
6 lies in between 1 and 1/2^, there is uncertainty as to the strategy that players 
will choose: for ^ > 1/2, because no symmetric Nash equilibrium in pure strate- 
gies exists when 1/2^ < 6 < 1; for ^ < 1/2, because there is coexistence of both 
full cooperation and full defection in the range 1 < 5 < 1/2^. In the former case 
the mixed strategies Nash equilibrium that fills the gap has the expected behav- 
ior: the probability of cooperating increases with the reward; however, in the latter 
case the behavior of this Nash equilibrium is counterintuitive, as the probability of 
cooperating decreases with the reward. This phenomenon can be explained in the 
framework of evolutionary dynamics, where the binary game models pairwise inter- 
actions between individuals of a large population. In this context, it is well known 
that, under the replicator dynamics, the equilibrium in mixed strategies of the Stag- 
hunt game is unstable and separates the basins of attraction of the two equilibria 
in pure strategies (full defection and full cooperation). We will come back to this 
issue in Section [3] in a more general setting, where we study in detail the replicator 
dynamics by considering interactions in groups of n individuals. 

Let us now analyze the case n>3. Notice that /Jn-i{q) defined in ^ is now a non- 
linear function of q and thus there can be more than one solution of fc{q) = foil)- 
However, as such solutions are obtained as the intersection points of a straight line 
with a strictly convex function, there can be up to two equilibria in the open interval 
(0, 1). As is proven in Theorem[T]of Appendix|Al the number of equilibria depends 
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(;>i/2 



MnC, 




C<1/n 
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Fig. 2. Symmetric Nash equilibria of the ?i-player game (n > 3) as a function of 5 for the 
three types of possible behavior, ^ > 1/2 (left) and l/?i<^<l/2 (middle) and C, < l/n 
(right). 

only on the values of 5 and C,. Moreover, the changes on the structure of equilibria 
when 6 increases correspond to three possible scenarios, determined by ^ < 1/n, 
l/n<^<l/2 and ^ > 1/2. (Notice that for n = 2 the middle case is empty, and the 
other two cases correspond to those discussed above.) Figure [2] depicts the typical 
structure of equilibria for these three cases. 

For the case ^ > 1/2, Theorem [H shows that there exists a unique symmetric Nash 
equilibrium which is a continuous increasing function of 5. It is strictly increasing 
within [1/n^, 1] from full defection at 5 = 1/n^ to full cooperation at 5 = 1, and 
constant outside the interval. However, when ^ < 1/2 we have two nontrivial, dif- 
ferent scenarios. One feature common to both of them is the existence of a range 
of rewards, namely max{l, l/nC} < 5 < 5^, for which two symmetric equilibria in 
mixed strategies coexist. One of these equilibria increases and the other decreases 
when the reward increases within this range. At the critical value 6c these equilibria 
collapse and a further increase in 6 yields a discontinuous jump from a Nash equi- 
librium with ^ < 1 to full cooperation. An upper bound for 6c is provided in Theo- 
rem [U The fundamental difference between the cases C,< 1/n and 1/n < C,< 1/2 
arises in the region min{ 1,1/ nC} < 6 < max{ 1,1/ nC,}, where there exists a unique 
equilibrium < q < 1: for l/n<^<l/2we see that q increases with 6, while 
for ^ < 1/n, we see that q decreases with 6, exhibiting the same counterintuitive 
behavior reported for the binary case. 

A case of particular importance is C, = 1, because it reproduces the cost/benefit 
parametrization of the PD game, by letting T = b, R = b — c, P = and S = —c, 
with b> c> 0. For this popular framework, suitable for biological applications, our 
result shows that the equilibrium of the shared reward dilemma only depends on the 
fixed amount p to be shared by the cooperators and on the cost c of cooperation, 
but it is independent of the benefit b. An analogou s result is observed in a spatial 
evolutionary version of the shared reward dilemma (jjimenez et Q/.Ll2007h . 



When the number of players n — > oo, we provide a simplified asymptotic version 
of Theorem [U in Corollary [T] of the Appendix |Al As in this limit the threshold 
1/n^ — > 0, the third of the three cases shown in Figure |2] disappears. Notice that 
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in order to get < 6 < 0° in the n — > 00 limit, we have to scale the reward with 
the number of interactions in the game, n{n— 1). The reason is that the payoffs 
collected per player from their pairwise interactions, in the first step of the game, 
are 0{n), therefore the reward per player must be of the same order to produce 
an effect. This makes p = 0{n^). In that case, the shapes of the first two cases in 
Figured are preserved, with a shift of the threshold to (full defection is an 
equilibrium if and only if p = o{rP-)). The critical value of the scaled reward, 6c-, at 
which the equilibrium jumps discontinuously from a value q<lio full cooperation 
when ^ < 1/2, can be exactly computed in the asymptotic case n ^ °°. As it is 
proved in Corollary [B 6^ = 1/4^(1 - Q. 

The limit case ^ +0° (equivalent to P ^ 5+) has a lso received special at t ention 



in the analysis of P D games on complex networks (|Nowak and SigmundL I200C ; 



Egufluz et g/.l |2005|) . Our results show (c.f. eq. (lA.ll) ) that a well defined mixed 
Nash equilibrium exists for < 6 < 1 which monotonically increases with 5 from 
to 1, reaching full cooperation for 5 > 1. In the n ^ 00 limit, using Corollary [U 
we can obtain an estimate for the equilibrium when P ^ S^, namely the smallest 
value between and 1 . 

Asymmetric Nash equilibria in pure strategies, in which part of the players in the 
group cooperate and the rest defect, can also be found for this game. For an inter- 
val of rewards starting at l/nC, (the maximum reward for which full defection is a 
Nash equilibrium) there exist asymmetric equilibria with k cooperators and n — k 
defectors. The value of k increases stepwise, starting from k = 1, at reward values 
= 5i < 82 < . . . (see eq. (|B.1I) ). with equilibria with k and k+l coopera- 
tors coexisting precisely and only at the separating values 5^^. For instance, upon 
increasing 5 above l/nC,, the full defection equilibrium is replaced by one with a 
single cooperator and n — 1 defectors. In turn, this is the only Nash equilibrium in 
pure strategies up 62, where it is replaced by another equilibrium with two coop- 
erators and n — 2 defectors. The maximum number of cooperators in asymmetric 
equilibria is n — 1 if C ^ 1/2^ or else the largest integer k < (n — 1)/2(1 — Q if 
^ < 1/2. In order to complete the analysis of the static game, a full characteriza- 
tion of these equilibria is given by Theorem [2] of Appendix |Bj There is a particular 
aspect of them which we would like to call attention upon: the fraction of cooper- 
ators in the asymmetric Nash equilibria approaches either the unique or the lowest 
mixed strategies Nash equilibrium < ^ < 1 in the limit « ^ 00. As we will see 
in Section [3l for the study of the replicator dynamics based on the shared reward 
dilemma, only the knowledge of symmetric Nash equilibria is necessary. 



3 Evolutionary dynamics 

In population dynamics, the evolution of coo peration can be modele d in several 



in population dynamics, tne evolution ol coo peration can be modele d in several 
ways. According to the replicator dynamics jHofbauer and Sigmund . [l998,) . the 
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dynamics in infinitely large populations is described by 



dx 

-=x(l-x)[/cW-/oW], (8) 

x{t) being the fraction of cooperators at time t and fc{x) and foix) the average fit- 
ness (which is the evolutionary counterpart of the concept of payoff) of cooperators 
and defector s in the population, r espectively. In this paper we consider the approach 



presented by iHauert et al.l (|2006f) to study replicator dynamics based on interaction 
groups of individuals. The standard setup to obtain the replicator equation is to 
assume a large population of individuals who randomly select partners to play a 
two-person game. In this alternative approach, players select groups of n — \ indi- 
viduals and play an n-person game instead. This is an appropriate approach to study 
the evolutionary beh avior of populations interacting through Public Goods games 



(IHauert et all l2006f) . and it is also suitable to study the evolutionary behavior of 



the shared reward dilemma. 

If the population is well-mixed, the number of cooperators at time t in an interaction 
group of n individuals is a binomial random variable with mean nx{t). Therefore, 
the average fitnesses at time t are given by formulae (U) and ([5]) with q = x{t). In- 
serting these formulae in ([8]) we model the evolution of cooperation when a reward 
p is available for each interaction group. 

It is clear that x = and x = I are always fixed points of the replicator equation dS]), 
but there will be further fixed points at the solutions of fc{x*) = foix*) in the open 
interval (0, 1). All of them are the symmetric Nash equilibria discussed in prev ious 
section. By the folk theorem of evolutionary game theory ( CressmanL 2003|) . the 



asymptotic stability of these fixed points will depend on the sign of fc{x) — foi^)- 
For example, if it is always positive, jc = is unstable whereas x = 1 is stable, 
and if it is always negative it is the other way around. The situation is different if 
fdx) — foix) changes sign in the interval (0, 1). By Theorem [T] (see Appendix [A]), 
we can determine how many roots (none, one or two) has fc{x) — foix) in the open 
interval (0, 1). On the other hand, since fc{0)-fD{0)=n{n-l){T-R){8-l /nQ, 
then .x: = is stable if 5 < l/nC, and it is unstable otherwise. Thus, we will find 
the following stability patterns, depending on the number of roots of (lA.ll) in the 
interval (0, 1): 

(I) if 6 < l/nC, (in this case there is either none or just one root), 

(a) if there are no roots, x = is a stable and x = I an unstable fixed point; 

(b) if there is one root < xi < I, then x = is a stable, xi an unstable and 
X = \ a stable fixed point, with xi separating the basins of attraction of 
x = and x= \ ; 

(II) if5> l/n^, 

(a) if there are no roots, x = is an unstable and x = I a stable fixed point; 

(b) if there is one root < xi < 1, then x = is an unstable, xi is a stable 
and a: = 1 an unstable fixed point; 
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Fig. 3. Equilibria of the replicator equation ([8]). Solid lines represent the asymptotically 
stable fixed points, while dashed lines represent the unstable ones. 

(c) if there are two roots < xi < X2 < 1, then x = is an unstable, xi a 
stable, X2 an unstable and x = I a stable fixed point, and X2 separates the 
basins of attraction of xi and x= I. 

All these situations are illustrated in Figure |3l Obviously the structure of fixed 
points of the replicator equation is the same as that of the symmetric Nash equilib- 
ria described in the previous section. The only difference is that now x = and x=l 
are always fixed points. What is really new is the stability patterns induced by the 
dynamics. These patterns are shown in Figure [3] through flux lines which indicate 
the direction in which the dynamics approaches the stable equilibria. It is worth 
noticing that for the two cases with C, < 1/2 (middle and right panels of Figure [3]) 
there is a critical value of the reward, 5^, at which, starting from a zero fraction of 
cooperators, the asymptotic cooperation level jumps discontinuously from a value 
^ < 1 to full cooperation. In both of them there is also a region of 5 in which, de- 
pending on the initial fraction of cooperators, the outcome may be full cooperation 
or a smaller fraction of cooperators. This smaller fraction outcome may even be 
in the case in which ^ < l/n. An important consequence is that, x = being unsta- 
ble for any 5 > l/nC,, for a suitable reward, a single mutant in an interaction group 
of defectors will spread cooperation in the population. 

To complete our analysis, we summarize the different dynamical regimes that can 
be obtained, by varying 5 and ^, in Fig.|4l These diagrams illustrate the transitions 
between the different evolutionary outcomes: full defection, coexistence of coop- 
erators and defectors, bi-stability — where full defection or full cooperation can be 
reached, depending on the initial population — , full cooperation, and — only for 
n>3 players — bi-stability between a mixed population and full cooperation. 



4 Conclusions 

In this paper we have studied the effect of rewarding cooperation in a strict social 
dilemma through the distribution of a fixed amount among all cooperative individ- 
uals. By adding this payment to the standard payoffs of the Prisoner's Dilemma, 
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Fig. 4. Diagrams sketching the different dynamical regimes for n = 2 (left) and n > 3 
(right) in terms of the two parameters C, (defection ratio) and 8 (scaled reward). Symbols 
stand for full defection (D), full cooperation (C), co-existence of defectors and cooperators 
in a mixed equilibrium (M), and existence of two stable equilibria, either full defection/full 
cooperation (D-C) or mixed equilibrium/full cooperation (M-C), each of which is reached 
depending on the fraction of cooperators in the initial population. The curve marking the 
upper bound for the D and D-C regions in both figures is given by ^ = l/n5. As n — > oo 
this curve moves towards the lower-left corner, thus shrinking these two regions, which 
disappear in the strict limit. The other curve of the right figure corresponds to the value of 
6 at which the two mixed equiUbria which are found for ^ < 1/2 coalesce (5c ; see text). 



cooperators and defectors in an interaction group confront a dilemma: on the one 
hand, individuals may be inclined to choose for shared reward despite the possibil- 
ity of being exploited by defectors; on the other hand, if too many players do that, 
cooperators will obtain a poor reward and defectors will outperform them. In the 
simplest case with only two players, we recover the traditional binary games for the 
study of cooperation where the social dilemma is relaxed: stag hunt and snowdrift. 

Although intuition suggests that in this game there should be a threshold value of 
the reward above which cooperation increases monotonically up to reaching satu- 
ration, the game exhibits more complex situations. The equilibrium structure has 
been characterized for the static game as well as for an evolutionary version of the 
game based on the replicator dynamics. For a wide range of parameters, scenar- 
ios with multiple interior equilibrium points are obtained, featuring critical values 
of the reward at which cooperation jumps discontinuously. Also, counterintuitive 
behavior where cooperation decreases as the reward increases may be observed. 
On the other hand, the replicator dynamics provides additional stability criteria 
for these equilibria. In the light of the stability patterns that arise, counterintuitive 
equilibria in the static game, exhibiting a decrease of cooperation upon increasing 
reward, turn out to be unstable equilibria of the dynamics separating basins of at- 
traction of other stable equilibria. As a consequence, a most relevant conclusion is 
that for many choices of the game parameters and initial conditions, the equilibrium 
with lower value of the cooperation level is dynamically selected instead of the full 
cooperation one. 
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The results presented in this paper allow for a complete characterization of the 
shared reward dilemma in the following terms. Cooperation does not appear until 
the reward increases above the threshold 5 = min{l, l/«^}. Interestingly, for 5 > 
1/n^, even a single cooperator can spread cooperation in the population, the more 
the larger the reward. This is an important point supporting the effe ctiveness of the 
reward mechanism for promoting the emergence of cooperation (|jimenez et al. . 
I2OO7 ). Subsequently, for ^ > 1/2 the fraction of cooperators increases monotoni- 
cally until full cooperation is reached for 6=1. However, and quite unexpectedly, 
for ^ < 1/2 an interesting phenomenon is observed: starting with a single coopera- 
tor, full invasion of the population only takes place when the scaled reward 6 > 5^, 
for some 5f > 1 . This resistance to cooperation is remarkable because for 5 > 1 full 
cooperation is a stable equilibrium of the dynamics, and agrees with the dynamical 
analysis that shows that full cooperation is only reached if the initial fraction of 
cooperators is already large. When crossing 5^ cooperation suddenly invades. At 
that point, if we decrease the reward again, full cooperation persists down to 5 = 1 . 
A slight decrease below this point produces an abrupt spread of defection in the 
population, which can even be completely invaded if ^ < l/n. This hysteresis is 
typical of critical phenomena, and it is very striking to find it in a model like this, 
where naive intuition says that the more one rewards cooperation, the more coop- 
erators should appear. The general, most important conclusion that can be drawn 
from this picture is that the effects of rewarding cooperation are neither trivial nor 
as straightforward as might be intuitively expected, and demand a more careful 
analysis. The origin of this complexity lies in the dilemma that the players con- 
front and the impossibility to know a priori how much reward a player can get by 
cooperating. 



One important issue for the shared reward dilemma is where this reward comes 
from. In the Introduction we have mentioned situations in Biology that can fit the 
setup of the shared reward dilemma, as well as mechanisms of direct rewarding to 
foster more social behavior. To name just one, companies have realized the need of 
searching for mechanisms that motivate, provide incentives or encourage coopera- 
tive behavior among their employees in order to contribute to the effective success 
of the teamwork. This context leads to another variant that we have not consid- 
ered here: the case in which the reward is detracted from the payoff of all players. 
This case is particularly interesting for two reasons: first of all, for the feedback 
mechanism that it implies, and secondly, because it models a common scenario of 
taxation and subsequent subsidy of only certain people. Given the complexity of 
the shared reward game as we have analyzed it here, the results of this new sce- 
nario are pres umed very rich. This tax-s ubsidy scenario has already been explored 
by some of us (|Lugo and Jimenez! |2006|) in a spatial evolutionary setup, but further, 
more detailed research is needed in view of the present findings. This issue will be 
the subject of a forthcoming work. 



In closing, we have shown that rewarding introduces a new social dilemma. De- 
pending on the parameters, the game casts the classical scenarios of full defection. 
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coexistence of cooperators and defectors, bi-stability of full defection and full co- 
operation, or full cooperation, as well as more complex scenarios with two interior 
mixed equilibria, where bi-stability between a mixed equilibrium and full coopera- 
tion can occur. In addition, we have seen that the cooperative response may not be 
continuous on the reward, implying that promoting cooperation may require sub- 
stantial incentives. We have shown that the classical (static) analysis of the game 
requires an evolutionary (dynamic) counterpart: while in the static case the counter- 
intuitive phenomenon of the decrease of the cooperation level upon increasing of 
the reward may occur, this is never found dynamically; on the other hand, in the 
evolutionary framework we observe that very large rewards may be needed to es- 
tablish a significant cooperation level, but once it is established, the reward may 
be very much reduced without damage to the cooperative behavior. Therefore, our 
general conclusion is that promoting cooperation through a reward mechanism is 
far from trivial, in agreement with the non trivial behavior found in many social 
contexts, and deserves careful consideration prior to, and during, application. 
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A Characterization of symmetric Nash equilibria 

Theorem 1 Let 5 = p/n{n — 1){T — R) be the scaled reward of the game and 
^ = {T —R)/(P — S) the defection ratio. Then, the following three scenarios can 
be found for the symmetric Nash equilibria of the shared reward dilemma with a 
number of players n>3>: 

1. For^> 1/2, 

(i) ifd < l/nC,, the unique Nash equilibrium is full defection (q — 0); 

(ii) 1/n^ < 5 < 1, the symmetric Nash equilibrium is a continuous function 
ofd which increases from 0"^ to 1, corresponding to the unique solution on 
(0, 1] of 

(C- 1>+ 1 - 5C = 0; (A.1) 



13 



(in) if5>l the unique Nash equilibrium is full cooperation (q = 1 j. 

2. For 1 /n < ^ < 1 /2, 

(i) ifd<l/nC, the only Nash equilibrium is full defection; 

(ii) if l/nCj < d < 1 the symmetric Nash equilibrium is a continuous function 
ofh which increases from 0^ to some limit smaller than 1, corresponding 
to the unique solution on (0, 1) o/ dA.iP ; 

(Hi) ifh>\ there exists he > 1 such that z/6 > 6c the unique Nash equilibrium 
is q = 1, whereas if 1 < 6 < 5^- there are two additional symmetric Nash 
equilibria corresponding to the solutions < qi <q2<l of ({A.l\) (equality, 
qi = q2 holds only for 6 = 8c). The equilibria q\ and q2 are continuous 
monotone functions ofd (increasing and decreasing respectively) and qi = 
1 when 5=1. 

3. For!;< \/n, 

(i) ifh< \ the only Nash equilibrium is full defection; 

(ii) 1 < 5 < 1/n^ the symmetric Nash equilibria are full defection, full co- 
operation and the unique solution on (0, 1] o/ i4.il) . which is a continuous 
function ofh which decreases from 1 to some limit greater than 0; 

(Hi) ifh >l/nC, there exists he > l/nC, such that ifh > h^ the unique Nash equi- 
librium is q = 1, whereas ifl<h<hc there are two additional symmetric 
Nash equilibria corresponding to the solutions < qi < q2 < I of 
(equality, qi = q2 holds only for h = he). The equilibria q\ and q2 are con- 
tinuous monotone functions ofh (increasing and decreasing respectively) 
and qi=0 when h= 1/nC,. 

An upper bound for he is given by 



Proof. As we discussed in Section Q, full cooperation is a Nash equilibria iff 
5 > 1 and full defection is iff 5 < l/nC,. To consider the remainder cases, let us 
define the "loss function" (|) : [0, 1] R, 




(A.2) 



fD{x)-fc{x) 

{n-\){P-S) 



4»iW-5C4»2W, 



(A.3) 



where (x) = x{^ — 1) + 1 and 




(A.4) 
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Fig. A.l. Relative situations of (^i{x) and 5C,(^2{x) (see text). 

(c.f. eq. (HI)-®). First of all, for 5 = the only root of the loss function is at 
X = 1/(1 — Q, which, for any ^ > 0, is outside the interval [0, 1]. Hence > 
for all jc G [0, 1] and the only Nash equilibrium if full defection. Let us henceforth 
assume 5 > 0. Function ^li^) decreases monotonically with x and, for any n> 2, 
is strictly convex within the interval [0,1]; instead, ^i{x) is a straight line with 
nonnegative or negative slope depending on whether ^ > 1 or ^ < 1, respectively. 
For reasons that will be clear in a while, we need to consider separately the cases 
^> 1,^< l/nandl/n<^< 1. 



Case ^ > 1 : 



As (^i{x) is nondecreasing, the loss function (|)(jc) monotonically increases with x 
and the only symmetric Nash equilibrium depends on the signs of (|)(0) = 1 — dC,n 
and^(l) = (1-5)C. 

(i) If 6 < 1/n^ we have < (|)(0) < (|)(1) and the unique Nash equilibrium is full 
defection. This equilibrium is strict for 5 < l/nC,. 

(ii) If 1/r^ < 6 < 1 we have (|)(0) < and (|)(1) > 0, and the symmetric Nash 
equilibrium in mixed strategies is the solution < < 1 of (lA.ll) . Note that 
(|)(jc) decreases with 5, thus q increases with 5. 

(iii) If 5 > 1 we have (|)(0) < (|)(1) < and the unique Nash equilibrium is full 
cooperation, which is strict for 5 > 1. 

In the next two cases ^ < 1 and therefore both ^i{x) and ^li^) are decreasing 
functions of x. As <^2{x) is convex, the situations that can occur are all sketched in 
fig.lAjJ 



Case C,< \/n: 



(i) If 5 < 1 then (|)(0) > and (|)(1) > and we have the situation sketched in 
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fig. lA.ir a). The only Nash equilibrium is full defection. 

(ii) If 1 < 5 < 1/n^ we have (|)(0) > and (|)(1) < 0, so the situation is as sketched 
in fig. lA.ir b) and therefore there will be a symmetric equilibrium < ^ < 1. 
Note that ^ = 1 for 5 = 1 and decreases as 5 goes to 1/n^. 

(iii) If 1/n^ < 6 then (|)(0) < and (|)(1) < 0. Thus we will have one of the two sit- 
uations plotted in figs. lA.U d) and lA.U e) depending on the slopes of <^i{x) 
and ^li^) at .x; = at the crossover 5 = l/nC,, where (|)(0) changes sign. 
If <^[{0) > <^2{0)/n the situation will be as illustrated in fig. lA.lT d). and if 
^'i (0) < <1^2(*^)/" ^i^^ ^S- lA.lf e). In the former case there will be 
two Nash equilibria, < qi < q2< 1, and in the latter the only Nash equilib- 
rium will be ^ = 1 . As (|)'^ (.)c) = ^ — 1 and 

we have (^\{0) = ^ - 1 and 4)^(0) = -n{n - I) /I. The condition > 
(^2(0) /n reads ^ > (3 — n) /2, which holds for any n > 3. We thus find two 
equilibria, < ^1 < ^2 < 1, vvhich, upon increasing 5, approach each other 
{qi increases and q2 decreases) up to 5c, where they coalesce in one Nash 
equilibrium q E (0, 1). Finally, for 5 > 5c the only Nash equilibrium is full 
cooperation. 

Case l/n <C,< 1 : 

(i) If 5 < l/nC, then (|)(0) > and (|)(1) > and we have the situation sketched in 
fjg. lA.U a). The only Nash equilibrium is again ^ = 0. 

(ii) If l/«^ < 5 < 1 (this case is empty if ^ = 1 /n) then (|)(0) < and (^(1) > 0, 
and we have the situation depicted in fjg. lA.lf c). There is a unique symmetric 
Nash equilibrium q E [0, 1) determined by (lA.ll) . Also ^ = for 5 = l/nC, and 
increases as 5 goes to 1 . 

(iii) If 5 > 1 then (|)(0) < and (|)(1) < 0. In this case we may have two additional 
equilibria if the situation of fig. lA.U d) occurs, or just one if either 5 > 1 
and we have the situation of fig. lA.lf e). or 5 = 1 and the situation is like 
in fig. lA.lf f). The separation between the first case and the last two cases 
depends on which scenario, fig. lA.lf d) or fig. lA.U f) we have at 5 = 1. This, 
in turn, depends on the slopes of ^i{x) and ^2{x) at .x; = 1 when 5 = 1: if 
fi(l) < ^^2(1) then we will have fig.lXHd), and if 4)i(l) > ^fjl^) we will 
have fig. lA.U f). The former is equivalent to ^ < 1/2, the latter to ^ > 1/2. So 
if ^ > 1/2 the only Nash equilibrium is ^ = 1, whereas if ^ < 1/2 there will 
be, for 1 < 5 < 5c, two equilibria, < ^1 < q2< 1, which coalesce in a single 
one at 5 = 5c. For 5 > 5c the only Nash equilibrium is ^ = 1 . 

The limiting value 5c can be determined as the value of 5 at which the curve {x) 
is tangent to hc'C,^2{x) at a point Xc E (0, 1). At this point the two equations 

C^l {Xc) = ?>c^<^2{Xc), C^l (Xc) = 5cC4'2(^c), (A.6) 
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hold simultaneously. These two equations can be combined to yield 



5eC(l -xr = xl{\-Q -x, + 6,C, (A.7) 
[(n - 1 ) - (n - 2) Qxj. - (n - 1+ 2Qxc + ^c^n = 0. (A.8) 

For Xc to exist it is necessary that the second equation has a solution. The condition 
for this to happen is 

(n - 1 + 2Q2 -4[(n _!)_(„_ 2)Qdc^n > 0. (A.9) 
Since ^ < 1/2 then (« — 1) — (n — 2)^ > 0, so the above equation holds provided 



4|(«-l)-(«-2)gC« 4C(l-=5fC) V « 
This expresses an upper bound for 5, 



Corollary 1 Consider a sequence {p„} of rewards such that pn ^ °° as n ^ °o in 
such a way that 

5=lim (A.11) 

with < 5 < oo. Le? w^' define 6^ = 1/4^(1 — Q. T/zen, m /imiY n the Nash 
equilibria of the shared reward dilemma are 

(i) full defection ifd = 0; 

( ii) a unique equilibrium in mixed strategies 



1- 

"= 20-0 

z/O < 6 < 1; 

(Hi) full cooperation and two equilibria in mixed strategies, Q < qi<q2<l, where 
qi is given by (E22]) cind 



1 + 1/1-8/81; 

2(1-0 ■ 

z/l < 5 < 5^ and ^ < 1/2 (equality qi =q2 = 1/2(1—^) only holds ifd — 5^j, 
and 

(iv) full cooperation otherwise. 

Proof.\T\ Asn^oo only two of the three cases of Theorem [Uremain, corresponding 
now to ^ > 1/2 and < ^ < 1/2. Besides, eq. (|A.1I) becomes the quadratic equation 

{C,-l)x^+x~bC, = 0, (A. 14) 
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whose two solutions are 



l-v/l-4C(l-05 l + v/l-4C(l-Q5 
^1 = ' ^2 = ^^3-3^^ . (A.15) 

Both are real whenever < 5 < 5^ = 1/4^(1 — Q. On the other hand, qi monoton- 
ically increases with 5. If ^ > 1/2, ^1 runs from to 1 as 5 moves from to 1; if 
^ < 1/2, qi goes from to 1/2(1 — Q as 5 goes from to 5^. As for q2, the condi- 
tion for it to be within the interval [0, 1] is ^ < 1/2 and 1 < 5 < 5^. When ^ = 1/2 
and 6=1 then q2 = qi = I. When ^ < 1/2 then q2 provides a second solution, 
monotonically decreasing from 1 down to 1/2(1 — Q as 5 runs from 1 to 5^, where 
it coalesces with qi. 

Finally, for 6 > 5^ we have 

(^-l)x2+jc-5^>0, (A.16) 
so the only Nash equilibrium is full cooperation. ■ 



B Characterization of asymmetric Nash equilibria 



Theorem 2 Let h = p / n{n — \){T — R) be the scaled reward of the game and C, = 
{T —R)/{P — S) the defection ratio. Let 

8, = ;i "-' + '*-''«-" . *=1,2,.. (B.l) 

n{n — IjC, 

Then a configuration with 1 < k < n — \ cooperators and n — k defectors will be 
a Nash equilibrium in pure strategies of the shared reward dilemma if and only if 
S/t < S < 5yt+i and, when C, < \/2, k < (n — 1)/2(1 — Q. 

Proof. According to ([T]), in a configuration with k cooperators and n — k defectors 
the payoff of a cooperator is 

S'c{k) = {k-\)R+{n-k)S + ^ (B.2) 

k 

and of a defector 

'PD{k)=kT + {n-\-k)P. (B.3) 

For such a configuration to be a Nash equilibrium in pure strategies two require- 
ments must be met: (i) a cooperator cannot get higher payoff by defecting, and (ii) a 
defector cannot get a higher payoff by cooperating. Condition (i) amounts to saying 
that (Pc{k) - ^ oik - I) > 0, i.e. 

{k-\){T-R) + {n-k){P-S)-^<0, (B.4) 
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and condition (ii) amounts to saying that 2'£)(^) — Tc{k + 1) > 0, i.e. 

k{T -R) + {n-\-k){P-S)--^ <Q. (B.5) 
By defining the parabola 

Vf{x)=x^{^-\) + {n-Qx-^, (B.6) 

where A = p/(/' — 5) = n(n — 1 )5^, and taking into account that P — 5 > 0, the two 
conditions above can be rewritten 

^{k)<0, \|/(yt+l)>0. (B.7) 

In other words, an asymmetric Nash equilibrium in pure strategies exists if and only 
if there exists k= 1,2,. — 1 such that (|B.7I) holds. 

The two roots of the parabola (IB. 61) are 

-(n-0±V(n-g2 + 4A(C-l) „, 
x± = , (B.8) 

so for the discussion to follow we should treat separately the cases ^ > 1, ^ = 1 and 
C<1. 

Case ^ > 1 . In this case the parabola is convex, both roots are real and x <0 and 
x^ > 0. So there will be an asymmetric Nash equilibrium in pure strategies with k 
cooperators if and only ifk<x+ <k+l, i.e. 



2fc(C - 1 ) < V (n - Q2 + 4A(C - 1 ) - (n - g < 2(fc + 1 ) (C - 1 ) (B.9) 
or equivalently 



{2k-l)!; + n-2k<^{n-Q^+4A{^-l) < {2k+l)!; + n-2{k+l). (B.IO) 

As ^ > 1 we have (2A: - 1)^ + n - 2A: > « - 1 > 0, so all three terms in (|B.10I) are 
positive numbers and can be squared to obtain, after simplifying, 

k[n~k+{k-l)Q <A< {k+ \){n-k- I +kQ. (B.ll) 

Given that A = — 1)6^, these inequalities can be rewritten 

8,<8<8,„. 8..^t "-' + '*-''«-" . (B.12) 

n[n — IjC, 

Notice that if ^ > 1 then {5yt} forms an increasing sequence and that 5i = 1/n^ and 
5„ = 1. 
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Case ^ = 1. In this case only the root = A/(n — 1) = r5 exists, thus the condition 
k <XQ<k-\-lh equivalent to (IB.12I) . where, of course, 5yt = k/n. 

Case ^ < 1 . The parabola (IB .61 ) is now concave and the roots can be rewritten 



_ (n-QTV(n-C)^-4A(l-C) 
x±- _Q • ^""-'^^ 

For them to be real we must have 

(n-Q2_4A(i-Q >0. (B.14) 

Suppose this inequality holds; then we have x±>0 and x^<X-. For an asymmetric 
Nash equilibrium with k cooperators to exist we must have k<x^ <k+l <x. 

The inequalities x+ < ^ + 1 < x_ are equivalent to 



\n~2k+{2k-\)l\ < y'(n-0^-4A(l-Q. (B.15) 

Squaring again this expression boils down to 5 < 5,t+i- The inequality k<x^ can 
be rewritten 



\n-Q^-A^{\-Q<n-2k+{2k-\% (B.16) 

No value of A satisfies this inequality unless the right-hand- side is nonnegative; in 
other words, unless 

*<5^. (B.17) 
Assuming (IB. 171) holds we can square and simplify once more to get 5 > 5yt. 

But there is one last remark to make: 5yt < 5 < 5yt+i is empty unless hk < ^k+i - If 
^ > 1 then 5yt is an increasing sequence, but for ^ < 1 this is no longer true, and the 
constraint 5yt < ^k+i implies 

*<^, (B.18) 

which is more restrictive than (|B.17I) . Notice that this only constraints the value of 
^ provided^ < 1/2. 

Finally, one can check that (IB. 141) holds for any dk because 

(^n-Q^-Anin~l)!;il-Qdk=[{2k-l){l-Q-n + lY>0. ■ (B.19) 
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